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ABSTRACT 

A saaple of 82 engineers and 82 supervisors rated the 
iaportance of 10 job perforaance criteria in relation to the 
successful perforaance of the engineer's job. Supervisors also rated 
the engineer's perforaance on each of the 10 specific criteria and 
tvo global aeasures. The ratings vere intercorr elated and factor 
analyzed; the extracted factors vere rotated using Kaiser's variaaz 
■ethod. The iaportance ratings by supervisors vere found to be highly 
similar to the iaportance ratings by engineers f with three factors 
emerging from the ratings by each group: (1) administrative skill, 
(2) technical competence, and (3) design skill. The factor analysis 
of the performance ratings by supervisors revealed tvo factors: (1) 
interpersonal relations ability and (2) technical ability. It appears 
that supervisors appraise performance by criteria similar to those 
they perceive as being critical to the type of work performed by 
engineers. (Author) 
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Ghiselli (1956) was one of the first to explicate the multidimen- 
sional nature of job performance. Studies by McQuitty, Wrigley, and 
Gaier (1954) , Seashore, Indik, and Georgopoulos (1960) and Turner (1960) 
served to validate the notion of multidimensional job performance criteria 
by identifying and isolating different dimensions of behavior related to 
job performance. While certain dimensions of job performance criteria 
have emerged from such studies, a second issue revolves around the extent 
to which differing functional work units within an organisation perceive 
these dimensions to be the same. 



their perceptions of the foreman's job responsibilities. A relatively 



larg«5 amount of disagreement was found between the average individual 
foreman and his own general foreman regarding the degree of responsibility 
the foreman had for different job specifications. Postulating that such 
disparate perceptions were in part a function of differing status levels 
in the organizational hierarchy, Lahiri (1965) and tahiri and C3ioudhuri 
(1966) studied two groups of employees (clerical and technical) who had 
the same job levels and salary grades. No overall differences were found 
between the perceptions of technical and clerical employees in regard to 
the relative importance of different job factors. Finally, Prien and 

♦Assistant Professor of Psychology, Iowa State University of Science 
and Technology, Ames, Iowa. 



Meyer (1959) studied how foreman and general foreman differed in 




Lisk© (1962) explored the relationship between supervisor ratings of job 
perfomance and incumbent ratings of job performance. Perceptions of 
performance criteria were found to vary consistently, with greater agree- 
nen: on intellectual elements of performance, but less with the social 
aspects of job perfoxmance. Prien and Liske concluded that, "Perceptions 
of performance criteria are in some way a function of status, or of proxi- 
mity to the actual performance being evaluated" (1962, p. 192). 

The purpose of the present study is to examine ti^ perceptions of a 
sample of engineers and their supervisors regarding tht importance of 
certain job performance criteria relating to the engineer's job. Super- 
visor ratings of the engineer's performance are also related to the per- 
ceived importance of the job performance criteria. 

METHOD 

Subjects 

The Ss were 82 male bachelor level engineering graduatfcs and 82 
supervisors. N«i«» of the engiu^ers had advanced academic degrees and all 
had 5-10 years of post college working experience. The sample of engineers 
had all graduated from the College of Engineering of Kansas State University 
and contained graduates of eight engineering departments (agricultural, 
architectural, chemical, civil, electrical, industrial, mechanical, and 
nuclear) . The sample is assumed to be representative of bachelor- level 
engineering graduates who have 5-10 years of post college working experience 



Procedure 

In consultation with College of Engineering officials, a list of ten 
criteria relevant to the successful performance of any engineeruig-related 
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job was devised. These were: (1) Scientific-Technical Knowledge; 
C2) Undexftanding of Engineering Problem-Solving Methodology; fS) Creativity- 
Originality; C4) Persuasiveness; (5) Interpersonal Cempetence; (6) Managerial 
Skill; (7) Written Comaunication; (8) Oral Conmunication; (9) Precision- 
Caie; aiid (10) Practical Judgment. Both the engineer and his supervisor 
were asked to rate the importance of each criterion to the successful per- 
formance of the engineer's job. A four-point scale was used: Essential 
(scored 4) ; Very Important (3) ; Important (2) ; Of Minor or No Importance (1) . 

Supervisors were also asked to rate the engineer's performance on each 
of the ten criteria. Five point rating scales were developed, with a 
different descriptive phrase defining the behavioral meaning of each point. 
The supervisor also was given the option to check "No Opportunity to Observe" 
for each criterion. For each scale, the most favorable rating was scored 5, 
the least favorable rating 1. In addition to these ten specific measures, 
tM^ global measures were included in the supervisor's rating. These were 
his rating of the overall quality and quantity of the engineer's work; 
again a five point rating scale was used. 

Analysis 

The Spearman rank order correlation was confuted between the mean 
importance ratings of the ten job performance criteria for the two rating 
groups. The importance ratings by both groups and the performance ratings 
by supervisors were intercorrelated and factor analyzed; the extracted 
factors were rotated using Kaiser's (1958) varimax method. 

RESULTS AND DISCUSSION 
Table 1 presents the means and standard deviations for the perceived 
importance ratings of the ten job performance criteria by both the super- 



4 



visors and engineers. In general, the two groups were in substantial agree- 
aent as to the importance of the ten j6b performance criteria. The Spear- 
man rank order correlation between the two sets of means in Table 1 was .69 

Insert Table 1 About Here 

(p< .03). Engineers as a group gave a statistically significant (p< .05) 
lower rating to Precision-Care than did supervisors. None of the other 
pairs of meani* differed significantly. Practical Judgment, Interpersonal 
Competence, and Oral Conmuni cation received respectively the highest mean 
ratings by both collective groups. However, examining the interrater 
reliability across each engineer- supervisor pair revealed the least amount 
of agreement (.03) on the importance of Practical Judgment, as can be seen 
in Table 2. 

Table 2 presents the intercorrelations between the rated importance 
of the ten job performance criteria, with the intercorrelation of the 
engineers* ratings below the diagonal, the intercorrelation of the super- 
visors' ratings above che diagonal, and the interrater reliability on the 
diagonal. Thus collectively, both engineers and supervisors rated Practical 
Judgment to be the most important criterion. However, there was virtually 

Insert Table 2 About Here 

no intenz/.ter agreement on this measure across pairs of raters. The 
greatest interrater reliability was for Scientific-Technical Knowledge (.47), 
which supports the findings of Px^en and Liske (1962) regarding greater 
agreement on intellectual measures of job performance. The modest size of 
the correlations between job performance criteria (Table 2) indicates a 
lack of dependence in importance between these measures. 
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Both sets of inportance ratings were factor analyzed and the rotated 
loadings are shown in Table 3. For the supervisors » the first factor has 

Insert Table 3 About Here 

strong positive loadings on Oral Communication, Persuasiveness, Written 
Comaunication, Managerial Skill, and Interpersonal Competence. !rhese per- 
formance measures clearly reflect the importance of administrative ability; 
the ability to plan, organize, communicate, etc. This factor is called 
"Administrative Skill." The second factor has strong positive loadings on 
Scientific-Technical Knowledge, Understanding of Problem-Solving Methodology, 

and Precision-Care. 

These performance measures reflect the importance of those skills that 
are typically engendered by formal academic training in engineering (see 
Hoyt and Muchinsky, 1970) ; factual knowledge, precise reasoning, tight 
thinking, etc. This second factor is called "Technical Competence." The 
third factor has strong positive loadings on Practical Judgment, Precision- 
Care, and Creativity-Originality. These performance measures reflect the 
importance of ability in the functional, operative aspects of engineering 
work, i.e., exercising sound judgment in the design and development of 
engineering projects. This third factor is called "Design Skill." 

For the engineers, the first factor has strong positive loadings on 
Oral Communication, Managerial Skill, Interpersonal Competence, Persuasive- 
ness, and Written Communication. These performance measures are the same as 
those in the first factor for supervisors, reflecting the importance of 
"Administrative Skill." The second factor has strong positive loadings on 
Creativity-Originality, Understanding of Problem-Solving Methodology, and 
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Practical Judgaent. Similar to the third factor for the supervisors, this 
factor is called "Design Skill." The third factor has strong positive 
loadings on Scientific-Technical Knowledge, Precision-Care, and Written 
CoBfflnini cation. These perfonaance criteria reflect the importance of 
engineering expertise. This factor is thus called "Technical Competence." 

In comparing the two factor structures it is evident that hoth super- 
visors and engineers have close common perceptions regarding the importance 
of the engineer's job performance criteria. The predominant factor in the 
ratings of both groups is the importance of administrative responsibilities. 
This factor for both groups has similar loadings on the same five job per- 
formance criteria. This factor, however, seemingly relates more to the job 
responsibilities of supervisors than engineers, as it reflects the impor- 
tance of managerial duties. However, both supervisors and engineers also 
perceived the importance of Technical Competence and Design Skill, factors 
which seemingly would relate more to the job responsibilities of engineers 
than the Administrative Skill factor. While the Technical Competence and 
Design Skill factors collectively accounted for more of the variance in 
the ratings than did the Administrative Skill factor, the latter emerged 
as the largest factor in both sets of ratings. 

These results do not support the f indingi of Meyer (1959) , who found 
considerable disagreement in perceptions of job responsibility between 
foreman and general foreman. The discrepancy between the findings of this 
study and Meyer's study may be due to differences in the working relation- 
ship between the two rating groups. Hoyt and Muchinsky (1970) reported 
that frequently supervisors direct the work of engineers vrfio are involved 
in team design projects, serving in both a supervisory and consultative 
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capacity. As Prien and Liske (1962) stated, if perceptions of performance 
criteria are a ftnction of proximity to the actual performance being 
evaluated, it may well be that the supervisors in this study had a closer 
working relationship with their subordinates than did the general foremen 
in Meyer's study. Therefore, the substantial degree of agreement between 
the two sets of importance ratings way be reflective of the fact that 
supervisors interact directly with engineers in performing their daily 
work operations. Such close proximity between the engineer and his super- 
visor nay eventuate in the evolution of common perceptions regarding the 
importance of certain job performance criteria. 

The emergence of an Administrative Skills factor in the engineer's 
job may be a function of the type of work performed by engineers. Engineers 
involved in team design projects are frequently called upon to communicate 
and "sell" their ideas to others, and to organize their individual talents 
into a cohesive design team (Hoyt and Muchinsky, 1970) . The importance 
of those skills is reflected in the high positive loadings on those per- 
formance criteria that comprise the Administrative Skills factor. Thus, 
while skill in administrative duties seemingly is more descriptive of a 
supervisory position, the particular nature of team design work may also 
engender the importance of these skills in the engineer's own position. 

The amount of variance in the importance ratings unaccounted for in 
the factor analyses is due probably, in part, to the unreliability of the 
rating instruments. Greater refinement of the importance rating scale 
would be desirable for future research, as the interrater agreement 
coefficients suggest somewhat of a ceiling effect in the importance ratings. 
However, it is clear from Table 3 that the three factors do emer^ from the 
importance ratings in the specific way shown. 
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Table 4 shows the intercorrelations of the engineer's perfoxmance 
ratings by stspervisors. All of the intercorrelations are positive and 

Insert Table 4 About Here 

most are only isodest in size, suggesting that these performance criteria 
reflect relatively independent aspects of an engineer's perforoance. Since 
only .01% of the performance ratings were marked "No Opportunity to Observe, 
the intercorrelations are not biased by frequent ratings of unobserved 
behavior. 

The performance ratings were factor analyzed and the rotated loadings 
are presented in Table 5. The first factor has strong positive loadings on 

Insert Table 4 About Here 

Interpersonal Competence* Oral Communication, Managerial Skill, Practical 
Judgment, and Persuasiveness. These loadings suggest the supervisors are 
evaluating the engineers* ability to work with the people-oriented problems 
associated with their job. Having the highest loading (.85) on Interper- 
sonal Competence and manifesting a ^xmlar pattern to the Administrative 
Skills factor from the importance ratings, this factor is called "Inter- 
personal Relations Ability," The second factor has strong positive load- 
ings on Creativity-Originality, Understanding Problem-Solving Methodology, 
Scientific-Technical Knowledge, Overall Quality, and Overall Quantity. 
These loadings suggest the supervisors are evaluating the technical and 
design merits of the engineer's performance, and is called "Technical 
Ability." 

In reviewing the performance ratings^ it appears that supervisors 
appraise the performance of engineers along similar lines to those by 



9 



which they perceive the importance of job perforaance criteria. I«iile 
there was not a hoaomorphic relationship between the importance and per- 
foxaance ratings, it seems the Administrative, Design, and Technical Skill 
factors of the importance ratings have their analog in the Interpersonal 
Relations and Technical Ability factors of the performance ratings. 

While both engineers and supervisors perceived the importance of three 
dimensions to the engineer's job performance criteria, the actual perfor- 
mance ratings by supervisors reflected appraisal on two dimensions. The 
importance of Administrative Skills in the engineer's job accounted for 
the greatest percent of the vailance in the ratings; likewise the correspond 
ing Interpersonal Relations Ability factor accounted for the greatest per- 
cent of variance in the performance ratings. While both supervisors and 
engineers could discriminate between the importance of technical and 
design skill, supervisors were apparently unable to make this discrimina- 
tion in their perfoimance ratings. Perhaps this stems from the difficulty 
of identifying and assessing creativity, as reflected in design projects 
(Taylor, 1964) . In general, however, supervisors appraised performance by 
criteria similar to those they perceived as being critical to the type of 
work performed by engineers. 
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Table 1 

Rated Importance of Job Performance Criteria 
For Performing Engineer's Job 



RaLln^ ^ Groups 





DupervibOrs 




Ctni^xnuers 






Mean 


S.D. 


Mean 


S.D. 


Practical Judgment 


3.51 


.61 


3.50 


.67 


Interpersonal Competence 


3.28 


.63 


3.43 


.61 


Oral Communication 


3.17 


.68 


3.28 


.67 


Precision-Care 


3.15 


.65 


2.87 


.75 


Managerial Skill 


3.11 


.7A 


3.21 


.77 


Understanding of Prob.-Solv. Methods 


3.02 


.85 


3.04 


.92 


Written Communication 


3.01 


.73 


3.07 


.78 


Scientific-Technical Knowledge 


2.94 


.88 


3.05 


.93 


Creativity-Originality 


2.91 


.72 


2.72 


.82 


Persuasiveness 


2.90 


.82 


3.06 


.82 
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Table 5 

Rotated Loadings from Factor Analysis of Engineer's 
Performance lutings by Supervisors 



Cumulative % of Variance 





32.1 


57.4 






I, 


II 




Scientific-Technical Knowledge 


.12 


.65 


.44 


Understanding of Problem-Solving Methods 


.28 


.67 


.53 


Creativi ty-Originali ty 


.06 


.78 


.61 


Persuasiveness 


.61 


.39 


.53 


Interpersonal Competence 


.85 • 


-.06 


.73 


Managerial Skill 


.63 


.52 


.67 


Written Communication 


.59 


.22 


.40 


Oral Communication 


.74 


.19 


.58 


Precision-Care 


.55 


.49 


.54 


Practical Judgment 


.62 


.45 


.58 


Overall Quality 


.44 


.69 


.67 


Overall Quantity 


.24 


.65 


.48 



